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Assistant Commissioner for Patents 

Washington, D.C. 20231 January 7, 2000 

Sir: 

The following preliminary amendments and remarks are 
respectfully submitted in connection with the above-identified 
application. 



IN THE SPECIFICATION; 

Page 1, line 6, delete "contents of"; same line 6, after 
"information" insert — , — ; same line 6, after "picture" 
insert — , — ; 

line 7, after "network" insert — ; and — ; same line 
7, after "particularly," insert — the invention relates — ; 

line 11, after "media" insert — , — (both 
occurrences) ; 
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line 15, delete " Art " insert — of the Invention — ; 

line 14, delete "the" insert — a — ; same line 14, 
delete "edited"; 

line 15, after "is" insert — edited so as to be — ; 
same line 15, after "a" insert — limited — ; same line 15, 
delete "band" insert — period — ; 

line 17, after "only" insert — that — ; same line 17, 
after "information" insert — that is — ; 

line 19, after "program" insert — it may be that — ; 
same line 19, delete "video" insert — new — ; 

line 20, after "to" insert — due to such editing — ; 

line 21, after "a" (second occurrence) insert — full 
length — ; same line 21, after "movie" (second occurrence) 
insert — as — . 

Page 2, line 1, delete "the" (first occurrence) insert 
— a — ; S ame line 1, after "program" insert — , which is — ; 

line 6, delete "additional data" insert — data in 
addition — ; 

line 7, after "signal" insert — period — ; 

line 10, after "language" insert — , — ; 

line 11, after ")" insert — , — ; same line 11, after 
"a" insert — video — ; 

line 12, delete "of a video signal" insert — , which 
is — ; same line 12, after "in" insert — the — ; 

line 13, after "broadcasted" insert — , — ; 
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line 14, after "time" (second occurrence) insert 
— used for this purpose — ; 

line 15, delete "an" insert — the — ; 

line 18, after "is" (second occurrence) insert 

line 19, delete "a computer" insert — the display 

unit — ; 

line 20, after "information" insert — , — ; 

line 23, after "information" (first occurrence) 
insert — , — ; same line 23, after "also" insert — is capable 
of—. 

Page 3, line 1, delete "the" insert — a — ; 

line 5, delete "a" (first occurrence) insert 

— one — ; 

line 6, after "data" insert — , — ; same line 6, 
after "information" insert — , — ; 

line 9, after "other" insert — ; and — ; same line 9, 
after "thus," insert — it is — ; same line 9, after "utilize" 
insert — the — ; 

line 10, delete "a" (second occurrence) ; 

line 11, after "information" insert — , — ; 

line 14, delete " Disclosure " insert — Summary — ; 

line 17, delete "so far as an" insert — in the form 

of—; 

line 18, delete "using a means"; 
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line 19, after "procedure" insert — , — . 
Page 4, line 3, delete "band" insert — period — ; 

line 9, delete "by referring to the following 
diagrams" insert — with reference to the accompanying 
drawings , — ; 

line 12, delete "shows the system configuration" 
insert — is a block diagram — ; 

line 15, after "3" insert — is a diagram which — ; 
line 17, after "4" insert — is a diagram which — ; 

line 22, delete "Fig. 7 shows" insert — Figs. 7(A) 
to 7(D) are diagrams which show — . 

Page 5, line 1, after "8" insert — is a diagram which — ; 
same line 1, delete "2" insert — an example of broadcast 
information, and Figs. 8(A) and 8(B) are diagrams which 
illustrate — ; 

line 4, delete "shows the system configuration" 
insert — is a block diagram — ; 

line 7, after "10" insert — is a diagram which — ; 

line 10, after "11" insert — is a diagram which — ; 

line 13, delete "explains" insert — will be referred 
to in order to explain — ; 

line 15, after "," insert — the — ; 

line 19, delete "the" (second occurrence) insert 
— an — ; same line 19, after "infrastructure" insert — , — ; 

line 21, after "information" insert — , — ; 
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line 23, after "and" insert — a — . 
Page 6, line 2, delete "to display" insert — and 
displays — ; 

line 7, after "203" insert — , — ; same line 7, after 
"respectively" insert — , — ; 

line 14, delete "to be the length of" insert — by — ; 
line 15, after "900" insert — to correspond to a 

length — ; 

line 16, after "a" insert — predetermined — ; 

line 18, delete ", to thereby" insert — , which is 

to — ; 

line 19, after "to" insert — various — ; 

line 21, after "means" insert — , — ; 

line 24, after "information" insert — and — ; 

line 25, delete "the" (first occurrence); same line 
25, delete "the" (second occurrence) insert — a — . 
Page 7, line 3, after "of" insert — the — ; 

line 10, delete "pair rental" insert — parental — ; 

line 11, delete "pair rental" insert — parental — ; 

line 12, after "work" insert — which may be — ; 

line 17, after "information" insert — , typically — ; 
same line 17, delete "typically"; 

line 19, delete "a time to watch and listen" insert 
— the time of viewing — ; 

line 20, delete "to"; same line 20, delete "pair 



5 



rental" insert — parental — ; 

line 22, after "Thus," insert — the — . 
Page 8, line 3, delete "To put it in detail" insert 
— More particularly — ; 

line 5, after "back" insert — properly — ; 

line 6, delete "properly"; 

line 8, after "to" (second occurrence) insert 
— the — ; same line 8, after "in" (second occurrence) insert 
— the — ; 

line 11, delete "capable of increasing the" insert 
— able to increase its — ; 

line 17, after "product" insert — , — ; 
line 18, after "program" insert — , — . 
Page 9, line 3, after "to" insert — the — ; 

line 4, after "program," insert — i.e. — ; 

line 9, after "in" insert — the — ; 

line 10, delete "a" (second occurrence) insert 

— the — ; 

line 12, delete "let" insert — allowed to — ; 
line 14, after "programs" insert — , — ; same line 
14, delete "an"; same line 14, delete "one"; 

line 15, after "operations" insert — , — ; 
line 18, delete "by itself"; 

line 19, delete "to see" insert — seeing — ; same 
line 19, delete "itself" insert — himself or herself — ; 



line 22, after "standardization" insert — , — ; 
line 23, delete "is making" insert — , has made — . 
Page 10, line 2, after "independently" insert — , — ; 

line 8, delete "merits" insert — advantages — ; same 
line 8, delete "reaped" insert — attained — ; 

line 9, delete "merits" insert — advantages — ; 
line 10, after "by" insert — a — ; 
line 12, after "in" insert — the — ; 

line 18, delete "resembling" insert — corresponding 
information recorded on — ; 

line 25, after "information" insert — , — . 
Page 11, line 4, after "objects" insert — , — ; 

line 20, after "106" insert — , — ; same line 20, 
after "drive" insert — , — . 

Page 12, line 4, after "108" insert — , — ; same line 4, 
after "monitor" insert — , — ; 

line 10, after "and" insert — for — ; 

line 14, after "medium" insert — , — ; 

line 15, after "MO" insert — , — ; 

line 16, after "medium" insert — , — ; same line 16, 
after "tape" insert — , — ; 

line 20, after "102" insert — , — ; 

line 21, after "processing" insert — , — ; same line 
21, after "decoding" insert — , — ; 

line 22, delete "by". 
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Page 13, line 6, after "that" insert — the — ; 

line 23, delete "of" (second occurrence) insert 

— in — ; 

line 24, after "information" insert — , — . 
Page 14, line 16, delete "only one" insert — single — . 
Page 16, line 20, delete "is" insert — will be — . 
Page 17, line 5, delete "is" insert — will be — ; same 
line 5, delete "by"; 

line 6, delete "referring" insert — with 
reference — ; 

line 10, after "means" insert — , — ; 

line 11, after "signal" (second occurrence) insert 

line 12, after "stream" insert — , — ; 

line 13, after "101" insert — , — ; 

line 16, after "or" (second occurrence) insert 

— to — . 

Page 18, line 14, after "ended" insert — , — ; 

line 15, delete "is" insert — will be — ; 

line 16, delete "by referring" insert — with 
reference — . 

Page 19, line 3, after "108" insert — , — ; 

line 7, after "107" insert — , — ; 

line 15, delete "As" insert — When — . 
Page 20, line 12, after "plat" insert — it — ; same line 



12, delete "it"; 

line 21, after "604" insert — , — ; same line 21, 
after "7C" insert — , — . 

Page 21, line 5, after "all" insert — of — . 
Page 22, line 25, after "all" insert — of — . 
Page 23, line 1, delete "As" insert — When — ; 

line 5, delete "As" insert — When — . 
Page 24, line 1, delete "an" insert — a — ; 
line 7, after "with" insert — the — ; 
line 8, delete "a" insert — the — ; 

line 17, delete "run fast" insert — fast forward — ; 

line 23, delete "there" insert — it — ; same line 2 3 
after "assumed" insert — that — ; same line 23, after "method" 
insert — is used — ; 

line 25, after "units" insert — , — . 
Page 26, line 5, delete "broadcast"; 

line 6, delete "at" insert — with — ; 

line 12, delete "are resulted in" insert — result — 

line 18, delete "be" insert — the — ; 

line 22, after "shows" insert — an example of the 
broadcast information, and Figs. 8A and 8B show — ; 

line 25, after "method" insert — , — . 
Page 27, line 6, after "denoted" insert — in Fig. 8 — ; 
same line 6, delete "till" insert — up to — . 

Page 28, line 10, delete "is" insert — will be — ; 
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line 13, delete "delivered" insert — provided — ; 
line 14, delete "using" insert — means of — ; 
line 15, delete " , " . 
Page 29, line 4, delete "is" insert — will be — ; same 
line 4, delete "as follows"; 

line 6, after "thereto" insert — , both of which 

are — ; 

line 13, after "and" insert — to — ; 

line 23, delete "band". 
Page 30, line 8, after "means" insert — , — ; same line 
delete "a" insert — the — ; 

line 9, after "embodiment" insert — , — ; 

line 10, delete "is" insert — will be — ; 

line 15, delete "the"; 

line 17, after "and" insert — is — . 
Page 31, line 5, delete 11 , adjusting" insert — and 
adjusts — ; 

line 9, delete "much" insert — greatly — ; 

line 10, after "out" insert — the — ; 

line 11, delete "a" insert — the — (both 
occurrences) ; 

line 18, delete "As" insert — When — ; 

line 21, after "back" insert — the — ; same line 2 
after "of" insert — the — ; 

line 24, after "is" insert — to be — . 
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Page 32, line 9, delete " Industrial Applicability "; 

line 14, after "using" insert — , — ; same line 14, 
after "example" insert — , — ; 

line 15, after "media" insert — , — . 



IN THE CLAIMS; 

Please amend the claims as follows: 
Claim 1, line 2, delete "band" insert — period — . 
Claim 3, line 4, after "and" insert — is — . 
Claim 4, line 11, after "with" insert — a — . 
Claim 5, line 4, delete "stopped. 
Claim 6, line 4, delete "stopped". 
Claim 8, line 11, after "with" insert — a — . 
Claim 9, line 2, after "if" insert — execution of — ; same 
line 2, delete "executed" insert — completed — ; 

line 3, after "advance" insert — further — . 
Claim 11, line 4, delete "band" insert — period — ; 

line 11, delete "of" insert — wherein — ; 

line 12, delete "interlocking"; same line 12, after 
"information" insert — is interlocked — ; 

line 14, after "with" insert — a — . 
Claim 12, line 6, after "with" insert — a — ; 

line 8, delete "to be" insert — and — ; 

line 10, delete "to be" insert — and — . 
Claim 14, line 4, after "with" insert — a — . 
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Claim 15, line 3, after "with" insert — a — ; 

line 4, delete "stopped"; 

line 10, delete "is" insert — are — . 
Claim 16, line 3, after "with" insert — a — ; 

line 4, delete "resuming" insert — resumes — ; same 
line 4, delete "stopped". 

Claim 18, line 2, after "if" insert — execution of — ; 

line 3, delete "executed" insert — completed — ; 

line 4, after "advance," insert — further — . 
Claim 20, line 4, after "determining" insert — a — ; 

IN THE ABSTRACT OF THE DISCLOSURE: 

Please replace the original abstract with the following 
new abstract: 
— ABSTRACT 

A broadcasting method and a broadcast signal receiving 
apparatus is provided wherein auxiliary information, such as 
video and audio data, for supplementing broadcast information 
presented through a broadcasting network in an interlocked 
manner can be watched and listed to by the viewer. Broadcast 
information received from a broadcasting network is 
temporarily stored in a storage means of a broadcast signal 
receiving apparatus and a recording medium for storing 
auxiliary information used as a supplement to the broadcast 
information is mounted on the broadcast signal receiving 
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apparatus. The broadcast information and the auxiliary 
information are then played back in an interlocked manner with 
a predetermined timing. As a result, a function to present 
information which can not be accommodated in a broadcast frame 
and a parental function can be implemented. — 

REMARKS 

The specification has been amended to correct errors of a 
typographical and grammatical nature. 

The claims have also been amended to more clearly 
describe the features of the present invention, and the 
abstract has been replaced by a new abstract, thereby 
correcting errors of a grammatical nature and providing an 
abstract in proper format. 

Also submitted herewith is a proposed amendment to the 
drawings, wherein Figs. 7 and 8 have been amended at this 
time. Upon receipt of the approval of the amendment to the 
drawings and receipt of a Notice of Allowance, the proposed 
drawing corrections will be effected in accordance with 
present practice. 

Entry of the preliminary amendments and examination of 
the application is respectfully requested. 

To the extent necessary, applicants petition for an 
extension of time under 37 CFR 1.136. Please charge any 
shortage in the fees due in connection with the filing of this 
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paper, including extension of time fees, to Deposit Account 
No. 01-2135 (501.37519X00) and please credit any excess fees 
to such deposit account. 

Respectfully submitted, 



CIB/DRA/cee 
(703) 312-6600 




Carl I. Bfirundidge 

Registration No. 29,621 

ANTONELLI, TERRY, STOUT & KRAUS, LLP 
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Serial No. : 



09/380,784 



Filed: 



September 9, 1999 



For: 



BROADCASTING METHOD AND BROADCAST SIGNAL 
RECEIVING APPARATUS 



Group : 



Examiner : 



PROPOSED AMENDMENT TO THE DRAWINGS 



Assistant Commissioner for Patents 
Washington, D.C. 20231 



January 7, 2 000 



Sir: 

It is proposed that the drawings in the above-identified 



lined print, and approval of these drawing corrections is 
respectfully requested at this time. 

Upon receipt of the approval of the amendment to the 
drawings and receipt of the official Notice of Allowance, the 
drawing amendments will be effected in accordance with the new 
procedures set forth in 1017 OG 4, April 6, 1982. 



application be amended in accordance with the attached red- 




Carl I . Birundidge 
Registration No. 29,621 
ANTONELLI, TERRY, STOUT & KRAUS , LLP 
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BROADCAS TING METHOD AND 



BROADCAST SIGNAL RECEIVING APPARATUS 



Technical Field 

The present invention relates to a broadcasting 
method and a broadcast signal receiving apparatus for 
presenting contents of video information such as a picture 
through a broadcasting network, more particularly, to a 
broadcasting method and a broadcast signal receiving 
apparatus wherein information presented through a 
broadcasting network and auxiliary information presented 
through other media such as package media can be watched and 
listened to in an interlocked manner. 

Background Art 

In the conventional system, edited video / audio 
information is broadcasted during a broadcasting band of 
time set as a program. Thus, the viewer can watch and listen 
to only part of such video / audio information actually 
broadcasted by a broadcasting station. For example, in the 
case of a news program, only part of gathered video 
information can be watched and listened to. In the case of 
a movie program, on the other hand, only part of a movie 
presented at a movie theater can be watched during the period 
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of the movie program usually shorter than the show time of 
the movie at the movie theater. 

Meanwhile, in recent years, there has been 
implemented a data broadcasting system whereby text data 
related to video / audio information is broadcasted as 
additional data to the video / audio information by 
appending the text data in the blanking signal of the video 
information. For example, systems called IT - vision and 
Intel - Intercast are known. In these data broadcasting 
systems, text data is described in a predetermined language 
such as an HTML (Hyper Text Markup Language) in a signal 
period of time of a video signal not used in video 
transmission to be broadcasted along with the video signal 
at the same time. An example of the signal period of time 
is an VBI (vertical blanking interval) . In the broadcast 
signal receiving apparatus, incoming video / audio 
information and text data are separated from each other. The 
video / audio information is played back as it is while the 
text data is displayed on a screen of a computer as a message 
related to the video information such as a text, a static 
picture or a graphic. In this way, the viewer is capable 
of not only watching and listening to the video / audio 
information but also obtaining information on its program 
at the same time. 
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In the data broadcasting system whereby auxiliary 
information is broadcasted by using the VBI with a low 
information transmission capacity, however, there is a 
limit on the amount of auxiliary information that can be 
transmitted at a time, making it impossible to broadcast a 
large amount of data such as video information as auxiliary 
information. While video information and related 
information are broadcasted at the same time, they are 
independent of each other, thus, difficult to utilize 
related information in a process of synthesizing a video 
information such as a search for original information based 
on the related information and correlation of a scene cut 
out from a movie with the original movie. 

Disclosure of the Invention 

It is an object of the present invention to provide 
the viewer with video / audio data which can not be included 
in broadcast information so far as an auxiliary information 
by using a means adopting a broadcasting procedure different 
from the normal broadcasting procedure wherein the 
auxiliary information is synthesized automatically in the 
broadcast signal receiving apparatus with the broadcast 
information to generate final information which appears to 
the viewer as if the final information had been broadcasted 
as a continuous single program. 
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According to the present invention, broadcast 
information received by the broadcast signal receiving 
apparatus during a predetermined program band of time is 
played back in a manner - interlocked with auxiliary 
information received in advance and held in the receiving 
apparatus . 

Brief Description of Drawings 

Embodiments of the present invention will be* 
described by referring to the following diagrams wherein: 

Fig. 1 schematically shows a package media 
interlocked broadcasting / watching / listening system; 

Fig. 2 shows the system configuration of a 1 st 
embodiment implementing a broadcast signal receiving 
apparatus; 

Fig. 3 shows the data format of broadcast information 
received by the 1 st embodiment ; 

Fig. 4 shows the data format of auxiliary information 
received by the 1 st embodiment; 

Fig. 5 shows a typical data format of an auxiliary 
information control table; 

Fig. 6 shows a typical video recording control table; 

Fig. 7 shows a sequence of user operations; 
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Fig. 8 shows 2 different algorithms adopted by a video 
synthesizing unit for synthesizing broadcast information 
and auxiliary information; 

Fig. 9 shows the system configuration of a 2 nd 
embodiment implementing a broadcast signal receiving 
apparatus ; 

Fig. 10 shows the data format of broadcast information 
and auxiliary information received by the 2 nd embodiment; 
and 

Fig. 11 shows the data format of auxiliary information 
handled in a 3 rd embodiment. 

Best Mode for Carrying Out the Invention 

Fig. 1 explains the concept of a package media 
interlocked broadcasting / watching / listening system. In 
the system, contents of a broadcast program are created by 
a contents provider 900 and the contents are partly 
delivered to a broadcasting station 9 01 as broadcast 
information 201. The broadcasting station 901 broadcasts 
the broadcast information 201 through the infrastructure 
such as a satellite, a ground wave facility or a CATV. The 
contents provider 900 also distributes other information 
related to the broadcast information 201, through the 
Internet and package media, as auxiliary information 301. 
The viewer side (at a home 9 02} receives the broadcast 
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information 201 and the auxiliary information 301 through 
a broadcast signal receiving apparatus 100 to display the 
information on a monitor. In the package media interlocked 
broadcasting / watching / listening system, auxiliary 
information 301 supplements broadcast information 201. 
Both auxiliary information 301 and broadcast information 
201 include indexes 305 and 203 respectively for associating 
the former with the latter and for specifying how and with 
what timing the former supplements the latter. 

The following is a description of an example to which 
the package media interlocked broadcasting / watching / 
listening system is applied. 

When distributing the contents of a large size theater 
movie, scenes in the movie are cut out to be the length of 
the broadcast information contents provider 900 adjusted to 
a broadcast time. The contents provider 9 00 delivers the 
movie-through-the-cutting-process to a broadcasting 
station, as broadcast information, to thereby be 
broadcasted to homes. On the other hand, the cut out scenes 
are sent to the homes as auxiliary information through other 
means such as the Internet, package media and data 
broadcasting systems. A broadcast signal receiving 
apparatus at each home receives the broadcast information 
and the auxiliary information, thereafter, displays both 
the pieces of information on a monitor in the interlocked 
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manner based on indexes appended to both. This enables the 
viewer to watch and listen to the contents of the complete 
movie. In a news program, part of news not broadcasted can 
also be presented to the viewer as auxiliary information. 
As a result, there is born a new information distribution 
business for distributing and delivering more detailed 
subjects, which are not broadcasted as a news program, as 
auxiliary information in the way described above. 

The package media interlocked broadcasting / watching 
/ listening system can also be applied to pair rental 
control. A pair rental control function prevents pictures 
and expressions of the original work inappropriate for 
juvenile viewers from being displayed. To be more specific, 
portions of the original work for general viewers are 
broadcasted as broadcast information whereas scenes for 
adult viewers are only delivered separately as auxiliary 
information by using typically package media. This allows 
the viewer to select whether or not such auxiliary 
information is to be displayed at a time to watch and listen 
to a broadcast program, implementing the pair rental control 
function to appropriately exclude such scenes from 
displayed information. Thus, chances that scenes 
inappropriate for juvenile viewers are inadvertently 
watched and listened to are eliminated. 
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Moreover, the package media interlocked broadcasting 
/ watching / listening system can also be applied to 
broadcast advertisements (CMs) . To put it in detail, by 
delivering a CM such as a product advertisement as auxiliary 
information, the CM can be played back in synchronization 
with a program properly. As a result, the number of 
opportunities in which a product advertisement is presented 
to the viewer in addition to CMs included in broadcast 
information can be increased. By setting an auxiliary 
information CM time in addition to the traditional CM time, 
the broadcasting station is capable of increasing the 
advertisement revenue. In addition, the CM provider is 
capable of appending a CM relevant to a program to the 
program broadcasted by a broadcasting station that does not 
broadcast the CM. For example, an index of 'the name of a 
program' included in broadcast information can be used to 
play back an advertisement of a product such as a car 
appearing in the program from a CD-ROM with timing specified 
by the index. In this way, the CM provider is capable of 
letting the viewer watch and listen to a CM even if the 
broadcast station does not broadcast the CM. 

The following is a description of an application of 
computer graphics to the package media interlocked 
broadcasting / watching / listening system. If the playback 
apparatus includes a processor that is capable of executing 
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a program, auxiliary information played back in a manner 
interlocked with an operation to display broadcast 
information will not be limited to video and .audio data 
described above. For example, a CG program, a program 
displaying computer graphics (CG) , can be provided as 
auxiliary information. That is, a CG program can be 
delivered as auxiliary information by using package media. 
Invocation of a CG program is triggered by index information 
included in broadcast information. By providing a parameter 
required for a CG program from a broadcast information, a 
CG character moving synchronously with the broadcasting is 
let appear on a displayed screen of the broadcast 
information. Such an application is particularly effective 
for programs such as an animation one. The viewer is capable 
of carrying out operations such as replacing some characters 
appearing in an animation program with CG characters 
selected from package media. If the viewer is also capable 
of creating a CG character by itself, the viewer will be able 
to enjoy to see a CG character created by itself appearing 
and acting in a broadcasted program. Such a joy can not be 
realized by the conventional TV broadcasting. 

In the mean time, standardization including one known 
as MPEG4 is making progress in recent years. In the video 
and audio encoding field, a technology called content based 
encoding has been implemented. The content based encoding 
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is a technology wherein video materials (or objects) 
included in video data is encoded independently and, on the 
receiving side, the objects are decoded individually and 
finally synthesized before being displayed. In this 
technology, since only information on objects that change 
in the video data is transmitted, not only is it possible 
to considerably reduce the amount of information to be 
transmitted, but other merits can also be reaped by a video 
data producer. The other merits include a capability of 
creating entirely new video data by combination of pieces 
of video data of objects photographed individually. With 
this content based encoding used in transmission of 
broadcast information, if the package media interlocked 
broadcasting / watching / listening system using a broadcast 
signal receiving apparatus provided by the present 
invention is adopted, video data of some objects included 
in a broadcast signal can be replaced by video data of other 
objects recorded in a piece of exactly resembling package 
media . 

For example, video data of a 1 st person appearing in 
a program and video data of a 2 nd person other than the 1 st 
person are each subjected to content base encoding with the 
2 nd person closely imitating behaviors of the 1 st person. The 
encoded data of the 1 st person is broadcasted as part of 
broadcast information while the encoded video data of the 
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2 nd person is recorded on package media for distribution. 
If only the broadcast information is received and played 
back, the 1 st person will appear in a program as it is along 
with other objects such as a background. If the broadcast 
information is played back in a manner interlocked with the 
package media, on the other hand, the viewer will be capable 
of watching and listening to video data wherein the 1 st 
person is replaced by the 2 nd person. 

The following is a description of a broadcast 'signal 
receiving apparatus of the present invention which makes the 
package media interlocked broadcasting / watching / 
listening system described above feasible. 

Fig. 2 shows the system configuration of a 1 st 
embodiment implementing a broadcast signal receiving 
apparatus 100 provided by the present invention. As shown 
in the figure, the broadcast signal receiving apparatus 100 
comprises : 

a broadcast signal receiving means 101 for receiving 
broadcast information 201; 

a storage means 10 6 such as a hard disc drive for 
temporarily storing broadcast information 201 received by 
the broadcast signal receiving means; 

a removable media drive 104 for inputting auxiliary 
information 301; 

an input means 107 for inputting a user intention; 
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a processor 102 for carrying out processing to convert 
broadcast information 201 and auxiliary information 301 
into a video signal; 

a display unit 108 such as a color monitor for 
displaying information ; 

a display means 103 for converting video data 
generated by the processor 102 into a signal to be displayed 
on the display unit 108; and 

a memory unit 105 used for storing programs 'to be 
executed by the processor 102 and temporarily storing data 
in the course of signal processing carried out by the 
processor 102. 

While it is desirable to put a mountable as well 
as removable random - access medium such as a CD - ROM, a 
DVD and an MO in the removable media drive 104, a sequential 
medium such as a tape is also acceptable as well. 

Broadcast information 201 is stored in the storage 
means 106 in program units. When the viewer watches and 
listens to a program, broadcast information 2 01 stored in 
the storage means 106 is transferred to the processor 102 
which then carries out processing such as decoding on the 
broadcast information 201 by using the memory unit 105 as 
a work area. A result of the processing is then output to 
the display means 103. If an index is extracted from the 
broadcast information 201 during the processing, auxiliary 
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inf ormation 3 01 indicated by the index is read out from a 
medium mounted on the removable media drive 104 . Processing 
is then carried out on the auxiliary information 301 and a 
result of the processing is output also to the display means 
103 . 

It should be noted that means used for furnishing 
auxiliary information 3 01 are not limited to media such as 
a CD - ROM. Auxiliary information 301 can also be supplied 
to the broadcast signal receiving apparatus 100 by'way of 
a network such as the Internet or a data broadcasting system 
such as a satellite broadcasting system or a ground wave 
broadcasting system. In this case, auxiliary information 
3 01 received from a partner on the Internet or a broadcasting 
station of the data broadcasting system is transferred to 
the storage means 106 or the memory unit 105 shown in Fig. 
2 to be stored therein. In either case, the auxiliary 
information 3 01 stored in the recording means 106 or the 
memory unit 105 is played back in a manner interlocked with 
broadcast information 201. 

Fig. 3 shows the data format of broadcast information 
201 received by the 1 st embodiment. As shown in the figure, 
a plurality of program indexes 203 are inserted between 
pieces of video and audio data of each program. Each of the 
program indexes 2 03 includes information such as a program 
ID 2 04 for uniquely identifying a broadcast program, a 
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program name 2 05 given to the program, a start time 206 of 
the program, an end time 2 07 of the program and one or more 
auxiliary information IDs 208 each for identifying a piece 
of auxiliary information 301 to be interlocked during an 
operation to play back the program. In the case of an analog 
broadcast signal, a program index 203 is embedded in a VBI 
region of the video signal by using the ordinary NTSC system 
video signal. In the case of a digital broadcast signal, 
on the other hand, a program index 2 03 is broadcasted as a 
control packet other than the video signal. 

It is desirable to insert a plurality of program 
indexes 203 into a program. This is because, if a program 
index 203 is inserted into only the head of a program, no 
program index 2 03 can be acquired in a recording or playback 
operation of the program starting from a position after the 
only one program index 203 , making it impossible to play back 
the program in a mode interlocked with auxiliary information 
301 with the auxiliary information 301 used as a supplement 
to broadcast information 201. By inserting a plurality of 
program indexes 2 03 into a program, however, this problem 
can be solved. A program index 2 03 may include auxiliary 
information IDs 208 of all pieces of auxiliary information 
3 01 to be interlocked with the program or include auxiliary 
information IDs 208 of only pieces of auxiliary information 
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301 to be interlocked with the program starting from a point 
into which the program index 2 03 is inserted . 

Fig. 4 shows the data format of auxiliary information 
301 received by the 1 st embodiment. As shown in the figure, 
auxiliary information 3 01 comprises a plurality of 
individual auxiliary information pieces 3 04 and an 
auxiliary information control table 303 for centrally 
controlling the individual auxiliary information pieces 
304. Each of the individual auxiliary information 'pieces 
3 04 comprises audio / video data 306 and an auxiliary 
information index 305 including information for identifying 
a partner program to be supplemented by the video / audio 
data 306. The auxiliary information index 305 comprises an 
auxiliary information ID 3 07 for distinguishing the 
individual auxiliary information pieces 304 from each 
other, the title 308 of the individual auxiliary information 
piece 304, a partner program ID 3 09 for identifying a partner 
program to be supplemented by the video / audio data 306, 
a supplementation start time 310 with the beginning of a 
partner program to be supplemented taken as a base, a 
supplementation end time 311 and time length information 312 
of the individual auxiliary information piece 304. 

Fig. 5 shows a typical data format of the auxiliary 
information control table 303. As shown in the figure, the 
auxiliary information control table 303 is a single 
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collection of attributes of auxiliary information indexes 
305 of all individual auxiliary information pieces 304 and 
storage locations 401. That is, each line of the auxiliary 
information control table 303 comprises attributes of an 
auxiliary information index 3 05 and one storage location 
401. A storage location 401 is an address in a medium or 
a storage device at which the video / audio data 306 of the 
individual auxiliary information piece 304 is stored. The 
auxiliary information indexes 305 cataloged in the 
auxiliary information control table 3 03 are sorted by 
partner ID 309. Note that it is desirable to automatically 
copy the auxiliary information table 303 into the storage 
means 306 at the time the medium storing the auxiliary 
information table 303 is mounted on the removable media 
drive 104. With the auxiliary information control table 303 
copied into the storage means 106, a search of the table 303 
for a desired piece of information can be carried out at a 
high speed. 

Next, a package media interlocked broadcasting / 
watching / listening method is explained. The package media 
interlocked broadcasting / watching / listening method 
comprises the steps of: 

carrying out a recording operation to record 
broadcast information 201 into the storage means 106; and 
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carrying out a synthesis / playback operation to play 
back a program while supplementing the broadcast 
information 201 with individual auxiliary information 
pieces 304. 

First of all, the recording operation is explained by 
referring to Figs. 2 to 6 . The viewer records broadcast 
information 201 into the storage means 106 in the same way 
as a TV program is recorded into a VTR (video tape recorder) . 
In the case of a storage means 106 implemented by a digital 
information storage means such as a hard disc, an analog 
broadcast signal is converted into a digital signal such as 
an MPEG stream by a video capture in the broadcast signal 
receiving means 101 and the resulting digital signal is then 
stored in the storage means 106. The viewer is allowed to 
specify a recording time in the case of reserved recording 
or unattended recording, or carry out a manual operation to 
start and end the recording. The processor 102 receives and 
stores incoming broadcast information 201 in accordance 
with a request coming from a timer not shown in the figure 
in the case of unattended recording or a request made by the 
viewer in the case of manual recording. At that time, the 
broadcast signal receiving apparatus 100 catalogs the 
recorded program into a video recording control table 501, 
a table for cataloging recorded programs , like one shown in 
Fig. 6. As shown in the figure, each line in the video 
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recording table 201 comprises a program ID 204 as well as 
a program name 205 extracted from a program index 203 of the 
broadcast information 201, a recording start time 504, a 
recording end time 505 and a recording location 506 or an 
address in the storage means 10 6 indicating the beginning 
of an area for storing the broadcast information 201. It 
should be noted that the recording start time 504 and the 
recording end time 505 shown in the video recording control 
table 501 are times with a program start time 206 "of the 
recorded program taken as a reference. That is to say, the 
recording start time 504 and the recording end time 505 are 
obtained by subtracting the program start time 206 from the 
time the recording is actually started and the time the 
recording is actually ended respectively. 

Next, a typical synthesis / playback operation is 
explained by referring to Figs . 2 to 7 . In order to obtain 
individual auxiliary information pieces 3 04 to be used for 
supplementing a program, a recording medium containing 
auxiliary information 301 is mounted on the removable media 
drive 104. When the recording operation described above is 
not carried out, the broadcast signal receiving apparatus 
100 carries out a through operation wherein the processor 
102 processes incoming broadcast information 201 and 
displays the processed broadcast information on the display 
unit 108 by way of the display means 103 as is the case with 
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an operation to receive an ordinary television broadcast 
program. For control purposes, the processor 102 displays 
a playback button 602 on the screen of the display unit 108 
as shown in Fig. 7A. 

When the viewer selects the playback button 602 by 
using a cursor 601 or presses a playback request button 
provided on the input means 107 such as a remote controller, 
the broadcast signal receiving apparatus 100 displays a 
recorded program menu 603 created at that time from the video 
recording control table 501 on the display unit 108 in order 
to let the viewer select a program recorded in the storage 
unit 10 6 to be played back. The viewer then selects a 
desired recorded program from the recorded program menu 603 
as shown in Fig. 7B by specifying the program name thereof. 
As the name of the desired recorded program is specified, 
the video recording control table 501 is searched for the 
name 205 of the desired program and the program ID 204 
thereof is acquired. 

Next, the processor 102 forms a decision as to whether 
or not auxiliary information 2 01 can be acquired from a 
recording medium mounted on the removable media driver 104 . 
The decision is formed by determining whether or not the 
program ID 2 04 of the selected program acquired from the 
video recording control table 501 exits as a partner program 
ID 309 in the auxiliary information control table 303 which 
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was copied from the recording medium into the storage means 
106 when the recording medium was mounted on the removable 
media drive 104 for the first time. If the outcome of the 
decision indicates that auxiliary information 2 01 can not 
be acquired, that is, if no partner program ID 309 in the 
auxiliary information control table 3 03 matches the program 
ID 204, supplementation of the program by auxiliary 
information 201 is ignored-. The processor 102 then searches 
the video recording control table 501 using the program ID 
204 as a key for the recording location 506. Subsequently, 
the processor 102 reads out the program from the location 
50 6 to play back it . If the outcome of the decision indicates 
that auxiliary information 201 can be acquired, that is, if 
a partner program ID 309 in the auxiliary information 
control table 303 matches the program ID 204, on the other 
hand, the matching partner program ID 3 09 is used for 
fetching attributes 307 to 312 and storage locations 401 of 
each auxiliary information index 3 05 associated with the 
desired program to be played back from the auxiliary 
information control table 303. An auxiliary information 
menu 604 shown in Fig. 7C is then displayed on the screen. 
It should be noted more than one auxiliary information index 
305 in the auxiliary information control table 3 03 may have 
a partner program ID 3 09 matching the program ID 2 04, that 
is, more than one individual auxiliary information piece 304 
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may be associated with the desired program. As shown in Fig. 
7C, the auxiliary information menu 604 includes a table of 
individual auxiliary information pieces 304 associated with 
the desired program and a 'Select all' button which is 
operated if it is desired to specify all the individual 
auxiliary information pieces 304. Beneath the auxiliary 
information menu 604, the following buttons 605 to 607 are 
also displayed on the screen shown in Fig. 7C. The 'Play 
back auxiliary information' button 605 is selected if it is 
desired to play back only the individual auxiliary 
information pieces 304. On the other hand, the 'Playback 
program' button 606 is selected if it is desired to play back 
the broadcast information 201 only, ignoring the individual 
auxiliary information pieces 304. Finally, the 'Synthesis 
/ Playback' button 607 is selected if it is desired to play 
back the program by supplementing the broadcast information 
201 of the program with the individual auxiliary information 
pieces 304. 

Assume that the viewer selects an 'auxiliary 
information 1' item representing one of the individual 
auxiliary information pieces 304 listed on the auxiliary 
information menu 604 and the 'Synthesis / Playback' button 
607. In this case, the processor 102 searches the video 
recording control table 501 using the program ID 204 as a 
key for a recording location 506, a recording start time 504 
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and a recording end time 505 of the specific broadcast 
information 2 01 to be played back. Then, the processor 102 
searches the auxiliary information control table 303 for a 
recording location 401 of the individual auxiliary 
information piece 3 04 corresponding to the 'auxiliary 
information 1' selected from the auxiliary information menu 
604 to be synthesized and played back with the specific 
broadcast information 201, a supplementation start time 310 
and a supplementation end time 311 of the individual 
auxiliary information piece 304 used in the synthesized 
playback. 

The recording start time 504 of the specific broadcast 
information 201 to be played back is compared with the 
supplementation start time 310 of the individual auxiliary 
information piece 304 to be synthesized. Since the 
recording start time 504 and the synthesis start time 310 
are both given with the program start time taken as a 
reference, the comparison can be decided to be justifiable. 
Video and audio data of the information with the smaller 
start time is played back as shown in Fig. 7D. 

An individual auxiliary information piece 3 04 is 
played back as follows. The processor 102 compares the 
supplementation start time 310 of the individual auxiliary 
information piece 304 with the contents of a timer also 
taking the program start time as a reference all the time. 
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As the contents of the timer become equal to the 
supplementation start time 310, the operation to play back 
the broadcast information 201 is temporarily suspended and 
an operation to play back the individual auxiliary 
information piece 304 is started. As the operation to play 
back the individual auxiliary information piece 3 04 is 
finished, the operation to play back the broadcast 
information 201 is resumed beginning with a portion at the 
supplementation start time 310 or the supplementation end 
time 311 of the individual auxiliary information piece 304. 
If the operation to play back the broadcast information 2 01 
is resumed beginning with a portion at the supplementation 
start time 310, the individual auxiliary information piece 
304 is merely inserted into the broadcast information 201. 
A method of synthesis to insert an individual auxiliary 
information piece 3 04 into broadcast information 2 01 is 
referred to as an insertion type synthesis method- If the 
operation to play back the broadcast information 201 is 
resumed beginning with a portion at the supplementation end 
time 311, on the other hand, the individual auxiliary 
information piece 304 replaces part of the broadcast 
information 201 between the supplementation start time 310 
and the supplementation end time 311 . A method of synthesis 
to replace part of broadcast information 201 with an 
individual auxiliary information piece 3 04 is referred to 
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as an replacement type synthesis method. The insertion type 
synthesis method and the replacement type synthesis method 
will be exemplified in concrete terms later. 

If the 'Play back auxiliary information' button 605 
shown in Fig. 7C is selected by the user, only individual 
auxiliary information pieces 3 04 are played back 
sequentially one piece after another beginning with one with 
a smallest supplementation start time 310 shown in the 
auxiliary information control table 303. If the 'Play back 
program' button 606 shown in Fig. 7C is selected by the user, 
on the other hand, only the broadcast information 201 
selected from the menu shown in Fig. 7B is played back. In 
addition, interface buttons 608 resembling buttons on a 
remote controller used for controlling an ordinary VTR are 
displayed on the screen as shown in Fig. 7D. The interface 
buttons 608 allow the user to specify operations to start, 
halt, temporarily halt, run fast and rewind the playback of 
the video data. When the user specifies the operation to 
halt the operation by operating one of the interface buttons 
608, the broadcast signal receiving apparatus 100 
terminates the operation to play back video data and resets 
the timer . 

In the above description, there is assumed a method 
whereby broadcast information 2 01 is recorded into the 
storage means 106 shown in Fig. 2 in program units and, in 
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an operation to play back a program, broadcast information 
201 of the program is synthesized with auxiliary information 
301. If the read / write speed of a hard disc drive used 
as the storage means 106 is not high enough to keep up with 
the recording and playback operations, however, part of the 
memory unit 105 can be used as a buffer between the processor 
102 and the storage means 106. The buffer allows the 
operations to record broadcast and play back information 2 01 
into and from the storage means 106 to appear as if they were 
carried out concurrently. As a result, the package media 
interlocked broadcasting / watching / listening method can 
be applied to a program which is currently being 
broadcasted. 

The buffer cited above works as follows. Broadcast 
information 201 received by the broadcast signal receiving 
means 101 is stored in a 1 st buffer area in the memory unit 
105. As the amount of stored broadcast information 201 
reaches a predetermined value, the broadcast information 
201 is transferred to the storage means 106 at one time as 
a batch. While the transfer is being carried out, the 
operation to store broadcast information 201 received by the 
broadcast signal receiving means 101 can be continued. In 
the mean time, broadcast information 2 01 transferred to and 
recorded in the storage means 106 is transferred back to a 
2 nd buffer area in the memory unit 105 in appropriate units 
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to be played back. Pieces of broadcast information 201 in 
the 2 nd buffer area are sequentially processed one piece 
after another by the processor 102 and output to the display 
means 103. In this way, while currently broadcasted 
broadcast information 201 is being recorded in the storage 
means 10 6, a playback operation can be continued at a time 
delay to a certain degree. While such simultaneous 
recording and playback operations are being carried out in 
this way, auxiliary information 3 01 can be used as" a 
supplement in accordance with program indexes included in 
the broadcast information 2 01 being played back. Time 
delays are resulted in due to supplementation by using 
auxiliary information 301. However, such time delays can 
be absorbed by the simultaneous recording and playback 
operations. If the insertion type synthesis method 
described above is taken as a technique of supplementation, 
the time delay between broadcast information being recorded 
and be information 2 01 being played back is widened. If the 
capacity of the storage means 106 is larger than a size 
corresponding to the time delay, however, there will be no 
missed information at all. 

Fig. 8 shows two different algorithms adopted in a 
synthesis of broadcast information 2 01 and auxiliary 
information 301. To be more specific, Fig. 8A is a diagram 
showing the insertion type synthesis method and Fig. 8B 
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shows the replacement type synthesis method. As shown in 
Fig. 8A, in the case of the insertion type synthesis method, 
the broadcast signal receiving unit 100 plays back auxiliary 
information 1 denoted by reference numeral 306 after playing 
back video and audio data 2 02 of broadcast information 2 01 
denoted by notation Ml till a point of time Tl . After the 
operation to play back auxiliary information 1 is finished, 
the operation to play back the video and audio data 202 of 
the broadcast information 201 is resumed starting with a 
portion M2 at the point of time Tl. In this way, auxiliary 
information 1 is merely inserted at the point of time Tl into 
the video and audio data 2 02 of the original broadcast 
information 201. In the case of the replacement type 
synthesis method shown in Fig. 8B, on the other hand, as the 
operation to playback the auxiliary information 1 is 
finished at a point of time T2 the operation to play back 
the video and audio data 2 02 of the broadcast information 
2 01 is resumed starting with a portion M3 at the point of 
time T2 . In this way, auxiliary information 1 replaces the 
portion M2 between the points of time Tl and T2 of the video 
and audio data 202 of the original broadcast information 
201. 

It should be noted that, in an effort made in the 
actual implementation to adjust timing in a playback 
operation, auxiliary information 301 is read out earlier 
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than an accurate start time and stored in a cache memory. 
By doing so, the auxiliary information 3 01 can be played back 
with good timing without causing the viewer to feel a sense 
of incompatibility. In addition, if interframe encoding is 
carried out to produce an MPEG stream, a frame on an encoding 
boundary, that is, an I frame in the case of MPEG encoding, 
is adjusted to coincide with a boundary between broadcast 
information 201 and auxiliary information 301. 

Next, a 2 nd embodiment implementing a broadcast* signal 
receiving apparatus provided by the present invention is 
explained. In the case of the 2 nd embodiment, auxiliary 
information 301 is also transmitted in the same way as 
broadcast information 2 01 instead of being delivered by 
using a recording medium. Fig. 9 shows the system 
configuration of the 2 nd embodiment. Since, auxiliary 
information 301 is also transmitted as is the case with 
broadcast information 2 01, as shown in the figure, the 2 nd 
embodiment is obtained by omitting the removable media drive 
104 from the system configuration of the 1 st embodiment shown 
in Fig. 2. Instead, an information separating program 801 
stored in the memory unit 105 is used for separating 
auxiliary information 301 from broadcast information 201 
received by the broadcast signal receiving means 101. A 
method for acquiring auxiliary information 301 in an 
operation to record video data is also different from that 
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adopted in the 1 st embodiment implementing the broadcast 
signal receiving apparatus 100 shown in Fig 2. 

An operation to record video data carried out in the 
2 nd embodiment is explained as follows . The broadcast signal 
receiving apparatus 100 receives broadcast information 201 
and auxiliary information 3 01 relevant thereto transmitted 
by a broadcasting station. Fig. 10 shows the data format 
of the broadcast information 201 and the auxiliary 
information 201 received by the 2 nd embodiment. The 
auxiliary information 3 01 is embedded in a program index 
203. The processor 102 executes the program 801 to extract 
auxiliary information 3 01 from information received by the 
broadcast signal receiving means 101 and store the 
information in the storage means 106. 

It should be noted that, in some cases, a large amount 
of auxiliary information 3 01 can not be transmitted by 
including the auxiliary information 301 in a program index 
2 03 of broadcast information 2 01as shown in Fig. 10 due to 
broadcasting specifications. In such a case, auxiliary 
information 301 is transmitted by splitting it into a 
plurality of packets. Typically, such packets are 
transmitted in advance independently of the associated 
broadcast information 201 prior to the time band of the 
program of the broadcast information 201. The processor 
102 employed in the broadcast signal receiving apparatus 100 
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stores auxiliary information 301 included in each packet. 
As the last packet is received, pieces of auxiliary 
information 3 01 are arranged in an order of sequence numbers 
included in the packets to reconstruct the entire auxiliary 
information 301. The 2 nd embodiment allows the viewer to 
automatically acquire auxiliary information 301 by 
receiving a broadcast signal without the need to resort to 
another means such as a recording medium used in the 1 st 
embodiment for distributing auxiliary information ' 3 01 . 

Next, a 3 rd embodiment of the present invention is 
explained. In the case of the 3 rd embodiment, the contents 
of auxiliary information 301 are a CG program, a sentence 
written in the HTML or an operation described as a script. 
In one typical version of the 3 rd embodiment, link 
information described in the HTML is received as auxiliary 
information 301. Video data generated by a source on the 
Internet is then acquired by tracing the link and displayed 
by executing a browser program. In another typical version 
of the 3 rd embodiment, a program for generating and 
displaying CGs (computer graphics) is received as auxiliary 
information 301. Then, by directly executing this CG 
program, CG video data can be displayed. Fig. 11 shows the 
data format of auxiliary information 301 handled by the 3 rd 
embodiment. As shown in the figure, an individual auxiliary 
information piece 304 comprises an auxiliary information 
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index 305 and a program or a script 1001. The auxiliary 
information index 305 includes a data type field 1002 
indicating the data type of the program or the script 1001. 
When executing a program or a script 1001, the processor 102 
refers to the data type field 1002, adjusting the execution 
of the program or the script 1001 to the data type specified 
in the field 1002. 

Unlike an operation to play back video and audio data, 
the execution time of a program or a script much depends on 
the performance of the processor 102 carrying out processing 
and the complexity of a network including a data source 
indicated by a link specified as auxiliary information 3 01. 
It is thus difficult to estimate the length of the execution 
time. For this reason, the auxiliary information index 3 05 
includes a processing time field 1003 . A positive value set 
in this processing time field 1003 is an effective period 
of time during which a result of execution of the program 
or the script 1001 is to be displayed. As the period of time 
lapses, the operation of the broadcast signal receiving 
apparatus is automatically switched to an operation to play 
back video and audio data 202 of broadcast information 201 
of the program. On the other hand, a negative value set in 
this processing time field 1003 indicates that the execution 
of the program or the script 1001 is continued till an 
instruction to end the execution is issued by the viewer. 
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Thus, in the case of a positive value set in this processing 
time field 1003, the processor 102 keeps monitoring the 
execution time while the program or the script 1002 of 
auxiliary information 201 is being executed. By virtue of 
the 3 rd embodiment, it becomes possible to provide the viewer 
with auxiliary information 301 including computer graphics, 
numbers , texts and figures in addition to video data by using 
various kinds of media. 

Industrial Applicability 

The present invention makes a package media 
interlocked broadcasting / watching / listening system 
feasible. In the package media interlocked broadcasting / 
watching / listening system auxiliary information relevant 
to broadcast information is distributed by using for example 
package media and, by playing back and displaying the 
auxiliary information and the broadcast information 
synchronously, the broadcast information can be 
supplemented with the auxiliary information. 



Claims 



1. (cancel) 

2. (cancel) 

3. (cancel) 

4. (amended) A broadcast signal receiving method whereby 
broadcast information broadcasted during a predetermined 
time band and received by a broadcast signal receiving 
apparatus is displayed in a manner interlocked with 
auxiliary information stored in said broadcast signal 
receiving apparatus in advance, 

wherein said broadcast information and said auxiliary 
information each include video and audio data, said method 
comprising the steps of: 

storing said video and audio data of received 
broadcast information in a storage unit employed in said 
broadcast signal receiving apparatus; 

playing back said video and audio data stored in said 
storage unit; and 

stopping the step of playing back said video and audio 
data stored in said storage unit with predetermined start 
timing and then playing back said video and audio data of 
said auxiliary information. 

5. A broadcast signal receiving method according to claim 
4 further comprising the step of stopping playing back said 
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video and audio data of said auxiliary information with 
predetermined end timing and then resuming the stopped step 
of playing back said video and audio data stored in said 
storage unit beginning with a portion of said video and audio 
data stored in said storage unit immediately succeeding a 
point of time corresponding to said predetermined start 
timing . 

6. A broadcast signal receiving method according to claim 
4 further comprising the step of stopping playing back said 
video and audio data of said auxiliary information with 
predetermined end timing and then resuming the stopped step 
of playing back said video and audio data stored in said 
storage unit beginning with a portion of said video and audio 
data stored in said storage unit immediately succeeding a 
point of time corresponding to said predetermined end 
timing. 

7. A broadcast signal receiving method according to claim 
4 whereby broadcast information currently being broadcasted 
and said auxiliary information are played back at the same 
time by concurrently storing said video and audio data of 
said currently broadcasted broadcast information in a 
storage unit and playing back said video and audio data 
stored in said storage unit. 

8. (cancel) 
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9. (amended) A broadcast signal receiving method whereby 
broadcast information broadcasted during a predetermined 
time band and received by a broadcast signal receiving 
apparatus is displayed in a manner interlocked with 
auxiliary information stored in said broadcast signal 
receiving apparatus in advance, 

wherein said broadcast information includes video and 
audio data whereas said auxiliary information includes a 
program or a script, said method comprising the steps of: 

storing said video and audio data of received 
broadcast information in a storage unit employed in said 
broadcast signal receiving apparatus; 

playing back said video and audio data stored in said 
storage unit; and 

stopping the step of playing back said video and audio 
data stored in said storage unit with predetermined start 
timing and then playing back said program or said script of 
said auxiliary information, 

whereby, if said program or said script is not executed in 
a processing period of time determined in advance, execution 
of said program or said script is canceled. 

10. (cancel) 

11. (amended) A broadcast signal receiving apparatus 
comprising : 




a storage unit for recording received broadcast 
information broadcasted during a predetermined program time 
band; 

a read unit for reading out auxiliary information 
from a recording medium mounted on said broadcast signal 
receiving apparatus to be played back in a manner 
interlocked with said broadcast information; and 
a processor for playing back said broadcast 
information recorded in said storage unit and said auxiliary 
information stored in said recording medium in a manner of 
interlocking said broadcast information with said auxiliary 
information by controlling read operations carried out by 
said read unit with predetermined timing, 

wherein broadcast information currently being 
broadcasted is played back in a manner interlocked with said 
auxiliary information by concurrently executing operations 
to record and play back said broadcast information into and 
from said storage unit. 

12. (amended) A broadcast signal receiving apparatus 
comprising: 

a storage unit for recording broadcast information 
and auxiliary information; and 

a processor for controlling an operation to read out 
broadcast information or auxiliary information from said 
storage unit with predetermined timing, 
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wherein said broadcast information is received 
through a broadcast system to be recorded in said storage 
unit and said auxiliary information is received through a 
broadcast system or a network to be recorded in said storage 
unit, 

wherein broadcast information currently being 
broadcasted is played back in a manner interlocked with said 
auxiliary information by concurrently executing operations 
to record and play back said broadcast information into and 
from said storage unit. 

13. (amended) A broadcast signal receiving apparatus 
comprising : 

a storage unit for recording received broadcast 
information broadcasted during a predetermined program time 
band; 

a read unit for reading out auxiliary information 
from a recording medium mounted on said broadcast signal 
receiving apparatus to be played back in a manner 
interlocked with said broadcast information; and 
a processor for playing back said broadcast 
information recorded in said storage unit and said auxiliary 
information stored in said recording medium in a manner of 
interlocking said broadcast information with said auxiliary 
information by controlling read operations carried out by 
said read unit with predetermined timing. 




wherein said processor stops an operation to play back 
video and audio data of said broadcast information with 
predetermined start timing and then plays back video and 
audio data of said auxiliary information. 

14. (amended) A broadcast signal receiving apparatus 
comprising: 

a storage unit for recording broadcast information 
and auxiliary information; and 

a processor for controlling an operation to read out 
broadcast information or auxiliary information from said 
storage unit with predetermined timing, 

wherein said broadcast information is received 
through a broadcast system to be recorded in said storage 
unit and said auxiliary information is received through a 
broadcast system or a network to be recorded in said storage 
unit, 

wherein said processor stops an operation to play 
back video and audio data of said broadcast information with 
predetermined start timing and then plays back video and 
audio data of said auxiliary information. 

15. (amended) A broadcast signal receiving apparatus 
according to claim 13 or claiml4 wherein said processor 
further stops playing back said video and audio data of said 
auxiliary information with predetermined end timing and 
then resumes said stopped operation to play back said video 
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and audio data of said broadcast information beginning with 
a portion of said video and audio data of said broadcast 
information immediately succeeding a point of time 
corresponding to said predetermined start timing so that 
said video and audio data of said auxiliary information is 
inserted into said broadcast information. 

16. (amended) A broadcast signal receiving apparatus 
according to claim 13 or claim 14 wherein said processor 
further stops playing back said video and audio data of said 
auxiliary information with predetermined end timing and 
then resuming said stopped operation to play back said video 
and audio data of said broadcast information beginning with 
a portion of said video and audio data of said broadcast 
information immediately succeeding a point of time 
corresponding to said predetermined end timing so that said 
video and audio data of said auxiliary information replaces 
said video and audio data of said broadcast information 
between said points of time corresponding to said 
predetermined start timing and said predetermined end 
timing . 

17. A broadcast signal receiving apparatus according to 
claim 11 or 12 wherein: 

said broadcast information includes video and audio 
data whereas said auxiliary information includes a program 
or a script; and 




said processor plays back said program or said script 
of said auxiliary information with predetermined start 
timing in a manner interlocked with said video and audio data 
stored in and then played back from said storage unit. 

18. A broadcast signal receiving apparatus according to 
claim 17 wherein, if said program or said script is not 
executed in a processing period of time determined in 
advance, execution of said program or said script is 
canceled. 

19. A recording medium for storing auxiliary information 
to be played back in a manner interlocked with a 
predetermined broadcast program wherein: 

said auxiliary information includes a plurality of 
individual auxiliary information pieces; 

the same plurality of individual auxiliary 
information pieces each comprise video and audio data of 
said individual auxiliary information piece to be played 
back in a manner interlocked with a program and an auxiliary 
information index including an auxiliary information ID for 
identifying said individual auxiliary information piece as 
well as a program ID for identifying said program with which 
said individual auxiliary information piece is to be 
interlocked. 

20. A recording medium according to claim 19 wherein said 
auxiliary information index of each of the same plurality 



of individual auxiliary information pieces also includes 
time information for determining timing with which an 
operation to play back said individual auxiliary 
information piece in a manner interlocked with said 
broadcast information is to be started and ended. 
21. A recording medium for storing auxiliary information 
to be played back in a manner interlocked with a 
predetermined broadcast program wherein: 

said auxiliary information includes a plurality of 
individual auxiliary information pieces; 

the same plurality of individual auxiliary 
information pieces each comprise a software program or a 
script of said individual auxiliary information piece to be 
played back in a manner interlocked with a broadcast program 
and an auxiliary information index including an auxiliary 
information ID for identifying said individual auxiliary 
information piece as well as a program ID for identifying 
said broadcast program with which said individual auxiliary 
information piece is to be interlocked. 
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Abstract 

The present invention relates to a broadcasting 
method and a broadcast signal receiving apparatus wherein 
auxiliary information such as video and audio data for 
supplementing broadcast information presented through a 
broadcasting network in an interlocked manner can be watched 
and listened to by the viewer. 

Broadcast information received from a broadcasting 
network is temporarily stored in a storage means of a 
broadcast signal receiving apparatus and a recording medium 
for storing auxiliary information used as a supplement to 
the broadcast information is mounted on the broadcast signal 
receiving apparatus. The broadcast information and the 
auxiliary information are then played back in an interlocked 
manner with predetermined timing. As a result, a function 
to present information which can not be accommodated in a 
broadcast frame and a pair rental function can be 
implemented. 
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